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Abstract 

The SINFONI data reduction pipeUne, as part of the ESO-VLT Data Flow System, provides 
recipes for Paranal Science Operations, and for Data Flow Operations at Garching headquarters. 
At Paranal, it is used for the quick-look data evaluation. For Data Flow Operations, it fulfills 
several functions: creating master calibrations; monitoring instrument health and data quality; 
and reducing science data for delivery to service mode users. The pipeline is available to the 
science community for reprocessing data with personalised reduction strategies and parameters. 
The pipeline recipes can be executed either with EsoRex at the command line level or through 
the Gasgano graphical user interface. The recipes are implemented with the ESO Common 
Pipeline Library (GPL). 

SINFONI is the Spectrograph for INtegral Field Observations in the Near Infrared (1.1-2.45 
/im) at the ESO-VLT. SINFONI was developed and build by ESO and MPE in collaboration 
with NOVA. It consists of the SPIFFI integral field spectrograph and an adaptive optics module 
which allows diffraction limited and seeing limited observations. The image sheer of SPIFFI 
chops the SINFONI field of view on the sky in 32 slices which are re- arranged to a pseudo slit. 
The latter is dispersed by one of the four possible gratings (J, H, K, H+K). The detector thus 
sees a spatial dimension (along the pseudo-slit) and a spectral dimension. 

We describe in this paper the main data reduction procedures of the SINFONI pipeUne, which 
is based on SPRED - the SPIFFI data reduction software developed by MPE, and the most 
recent developments after more than a year of SINFONI operations. 
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1. Introduction 

SINFONI is the Spectrograph for Integral Field Observations in the Near Infra Red 
(NIR) . It is attached to the Cassegrain focus of Yepun, the fourth Unit Telescope (UT4) 
on Cerro Paranal in northern Chile. It allows diffraction and seeing limited integral field 
observations in the near infrared (1.1-2.45 /im) at 3 spatial resolutions: 250, 100 and 25 
mas /pixel. 

We will provide a general introduction to instrument pipelines, then present the fea- 
tures of the Common Pipeline Library (CPL) and its front end applications Gasgano and 
EsoRex. These will be followed by an overview of the instrument and an introduction 
to the principle of integral field spectroscopy. Next we will describe the data reduction 
cascade and the main data reduction challenges. Finally we will highlight some scientific 
results based on pipeline reduced data products which were obtained during the science 
verification phase. 

2. Instrument Pipelines Objectives 

Today twelve instruments are operational at Paranal. They cover the observational 
domain over a wide range of spectral and spatial resolutions. ESO uses instrument data 
reduction pipelines to support the complex end to end data flow operation model. The 
main objectives of the pipeline are to: 

- Support the Garching Data Flow Operation (DFO) department in the: 

• Processing of raw calibration frames into master calibrations. 

• Generation of quality control (QCl) parameters to monitor telescope, instrument 
and detector performance, so that the quality of calibration and science data can be 
assessed in due time. 

• Reduction of sets of raw science data files to science data products. 

- Support the Paranal Science Operations (PSO) with a quick-look tool to assess the 
instrument health and the quality of the science and calibration data in due time. 

- Allow users to perform a personalised data reduction with user selected reduction 
strategies and parameter settings. 

The VLT instrument pipeline releases are publicly available at www. e so . org/pipe lines. 
More details are given in [Ballester et al.(2006)] and [Silva D. & Peron M. (2004)]. 

3. The Common Pipeline Library 

The Common Pipeline Library (CPL) has been produced by ESO to aid in the develop- 
ment of VLT pipelines with standardised implementations and lower maintenance costs. 
CPL is offered to instrument consortia and the user community to build instrument data 
reduction software. It is distributed as a development kit together with documentation 
and recipe front-ends. It is available on the web at www.eso.org/cpl. 

A CPL based pipeline is structured as a set of recipe plug-ins which are called by 
front-end applications (EsoRex or Gasgano). The recipes of the Data Reduction Software 
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Fig. 1. The Gasgano front end. On the left, the 
main panel with a selection of calibrations and 
raw data required to reduce a telluric standard 
star observation. On the right, the corresponding 













recipe execution control panel. 

(DRS) are implemented as CPL plug-ins - units of code which are dynamically loaded 
into a host application at run time. The parent application is shielded from the recipe 
plug-in's implementation details by the CPL plug-in interface. This interface consists of 
three functions, to initialize, execute and destroy the recipe plug-in. These functions need 
to be provided by the recipe implementation. 

The SINFONI pipeline is the CPL implementation of the Python based SPRED pack- 
age, the SPIFFI data reduction software developed by the Max-Planck-Institut fiir ex- 
traterrestrische Physik (MPE, see also [Abuter et al.(2006)] and [Schreiber et al.(2004)]). 

4. Pipeline Front-End Applications 

Gasgano is a data management tool that simplifies the data organisation process. It of- 
fers automatic data classification and association (Fig. 1). Gasgano classifies the raw and 
reduced data files based on their FITS keyword content by applying instrument specific 
rules. It allows users to execute recipes directly from a GUI with fiexible data reduction 
strategies. Gasgano is available from the ESO web pages at www.eso.org/gasgano. 

EsoRex, the ESO Recipe executor, allows the execution of recipes from the command 
line. It does not offer the high level functionalities of Gasgano, but it can easily be 
embedded in scripts. 

5. Integral Field Spectroscopy 

For SINFONI, the instrument's two dimensional field of view (FOV) is sliced into a 
one dimensional pseudo long-slit, which is then dispersed by a spectrometer onto the 
two dimensional detector array (see Fig. 2 for a simplified overview of the principle). 
The recorded data contains the spatially resolved image slices (in one direction) and 
the spectra (in the other). The data reduction software has the task of removing the 
instrument signature, in this case reconstructing the full spatial and spectral information 
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into a 3D data cube. Here, the FOV is mapped into the first and second dimensions, with 
the wavelength in the third dimension. 




Fig. 2. This figure describes the Integral Field Spectroscopy concept: a two dimensional image on the 
sky is sliced, dispersed and imaged on the detector. The data reduction software then reconstructs a 
cube containing the full spatial and spectral information. 



6. SINFONI 

SINFONI is composed of two subsystems: the Multi- Application Curvature Adaptive 
Optics (MACAO) unit, which allows diffraction and seeing limited observations, and 
SPIFFI, the spectrometer for Infrared Faint Field Imaging, an integral field spectrograph 
which is shown in Fig. 3. It is described with more details in [Bonnet et al.(2004)] and 
[Eisenhauer et al. (2003)]. 

Following the optical path, the light coming from the MACAO module is collimated by 
a doubled lens onto a cold stop to suppress the thermal background from the telescope 
secondary mirror. Then the light path continues through a broadband filter (to suppress 
unwanted diffraction orders of the grating) and the optics wheel hosting the camera lens 
systems for the three different image scales (250, 100, 25 mas/pixel). The images are 
focused on the small image sheer, a stack of 32 plane mirrors which slice the image into 
slitlets and separates the light from each slitlet into different directions. A second set of 
32 mirrors, the big sheer, collects the light and forms a pseudo-slit which is then dispersed 
by the selected grating (J, H, K and HK bands) and finally imaged by the camera to the 
2D NIR detector array. 

The instrument FOV is rearranged on the detector into 32 vertical strips, (slices) each 
64 pixels wide and 2048 pixels long, according to a given sequence in a brick- wall pattern. 
For SINFONI, the wavelength increases from the top to the bottom of the detector array. 
Sky lines appear as small horizontal stripes, whilst cosmic rays and hot pixels manifest 
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themselves as bright random spots (pixel aggregates). Dead pixels appear as black spots. 
Fig. 4 displays many of these features. 




Fig. 3. The SPIFFI spectrograph. Fig. 4. Typical data features. 



7. Data Reduction Cascade 

The SINFONI data reduction is described in more detail in the SINFONI Pipeline 
User Manual, available on the web (www.eso.org/pipelines). An overview of the data 
reduction cascade is shown in Fig. 5. The following recipes are used in the data reduction 
process: 

- The linearity recipe derives a map of non-linear pixels from a set of flat fields with 
increasing intensity. 

- The master dark recipe derives a master dark frame and a map of hot pixels from a 
set of dark frames. 

- The master fiat recipe derives a master fiat field frame and a map of pixels whose 
intensity is above a given threshold from a set of flat field frames. It combines this 
map with the input non-linear pixel map and the reference bad pixel map to build the 
master bad pixel map. 

- The distortion recipe determines the optical distortion coefficients for image recon- 
struction and the slitlets' distance tables from a set of (typically 75) fibre flats. Fibre 
flats are taken with the calibration fibre moved perpendicularly to the image slices. 
Additional inputs are a pair of on/off arc lamp frames, standard flats, a reference arc 
line list and a "setup" table. 

- The wavelength calibration recipe determines the wavelength map and the slitlet po- 
sition table from: a set of arc lamp frames, a master flat, a master bad pixel map, a 
distortion table, a reference arc line table, and a "setup" table. 

- The jitter recipe reduces standard star calibrators and science target observations at 
the bottom end of the data reduction cascade. It requires as input a master bad pixel 
map, a master flat, a master dark of the proper detector DIT, a distortion table, a 
wavelength map, a slitlet distances table and a slitlet position table. The recipe extracts 
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Fig. 5. Data reduction sequence with recipe and calibration product dependencies. 



the object spectrum (for sources whose Point Spread Function is weh approximated 
by a 2D Gaussian), computes the instrument's Strehl ratio, determines instrumental 
throughput and reconstructs a 3D cube. This cube is composed of a stack of planes, 
each of which contains the full spatial information of the instrument FOV. The wave- 
length information is mapped on the cube's third dimension. If the recipe is provided 
with several object frames - for example, a sequence of on/off-sky or jittered images - 
it will generate the sky subtracted mosaic data cube from the raw data frames (Fig. 
9). 



8. Pipeline challenges 

The main tasks of the SINFONI pipeline are: the calculation of the optical distortion 
coefficients, the computation of a wavelength calibration solution, the reconstruction 
of the full spatial and spectral information on a 3D data cube, and the removal of 
the emission from the telescope, the instrument and the sky (dominant in the NIR). 
Additionally, the pipeline should generate the information needed to monitor the behavior 
of the instrument as atmospheric conditions change. These QCl parameters are used by 
DFO to monitor the quality of master calibrations, and science products; they also give an 
indication of the instrument's health (see [Hummel et al.(2006)] for a more comprehensive 
overview) . 
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8.1. Distortion Computation: 



The distortion function, which characterises the curvature of the spectral traces of the 
32 shces, is calculated from a series of continuous light fibre spectra taken with the fibre 
moved perpendicularly to the slices. For every position of the fibre a raw FITS data file 
is recorded. With an initial data reduction step the single fibre spectra are co-added to 
an all-fibers synthetic frame and the offset of the first slitlet is calculated as a reference 
position. The distortion is computed in three further steps: 

- Detecting the fibres and tracing the curved fibre spectra. 

- Constructing two numerical grids on the distorted and the undistorted fibre positions. 

- Solving a 2D polynomial fit to transform positions from raw to undistorted coordinates. 
Finally, the recipe determines the position of the edges of the slitlets by fitting the edges 
of the brightest arc line of each slitlet with a linear step function or a Boltzmann function. 





FINE TUNING SLICES ALIGNMENT 



Fig. 6. This image describes the concept of the 
distortion computation algorithm. The spectra of 
the fibres are traced, two grids on the distorted and 
undistorted space are built, then a 2D polynomial 
transformation is performed. This figure displays 
only two of the 32 slitlets. 



Fig. 7. Cube reconstruction: raw data are re- 
sampled using a wavelength map to remove the 
brick-wall pattern. The slitlets are then stacked 
into a cube taking slitlet distances and edge po- 
sitions into account. Each plane of the cube is a 
monochromatic image of the instrument FOV. 



8.2. Wavelength Calibration: 

The wavelength calibration is based on a set of on/off arc lamp exposures. Initially, 
the arc lamp frames are stacked and the off-frames are subtracted from the on-frames 
to remove the thermal background. The resulting frame is fiat fielded, the distortion 
correction is applied, and static bad pixels are corrected. 

Using a polynomial model, guess values for the central wavelength Aq and the first 
and second order dispersion coefficients, it is possible to build a synthetic frame from 
the reference arc line list by associating at each catalog entry a delta-like function of 
proper wavelength and intensity. Then this spectrum is convolved with a Gaussian having 
a FWHM appropriate to the instrumental resolution. The synthetic frame thus has a 
well known wavelength and intensity. Cross correlation of the synthetic frame with the 
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observed spectrum provides an approximate wavelength and line intensity for each raw 
(and emission line). 

Doing a non-linear least squares fit with a Gaussian to the synthetic frame, one can as- 
sociate to each line an accurate intensity, position, FWHM and background. This permits 
the construction of line index triplets (x, A). After removal of false identifications, dis- 
persion coefficients are derived by a polynomial fit along each detector's column. Finally 
the smoothed wavelength dispersion coefficients are obtained with a low order polynomial 
interpolation of each dispersion coefficient in the detector's spatial direction. 



8.3. 3D Data Cube Reconstruction: 



From the input set of frames object-sky pairs are defined by associating to each ob- 
ject frame the sky frame that is closest in time. For each object-sky pair a 3D data 
cube is constructed as follows. If a master dark is provided as input of the jitter recipe, 
this is subtracted from the object and the sky frames, then the corrected sky frame is 
subtracted from the corrected object frame, and the resulting frame is flat fielded and 
distortion corrected. The resulting image is re-sampled into the same wavelength steps, a 
process which aligns the different slitlets in wavelength. A 3D data cube is subsequently 
constructed using the information read from the slitlet position table and the slitlet edge 
distances table. As not all of the slitlets are exactly 64 pixels long and 64 pixels dis- 
tant from the adjacent one, it is necessary to refine the slitlets' position alignment on 
the stacked cube. The jitter recipe then constructs a mosaic of the series of input data 
cubes (each aligned in the co-added space) by reading the cumulative offset information 
from the FITS headers of the raw input frames (Fig. 9). Finally, the recipe extracts the 
spectrum along the Z-axis. 
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Fig. 8. Comparison between un-corrected 
(black) and corrected (red) sky subtracted 
object spectra. 



Fig. 9. Observations of stars embedded in the giant HII 
regions W42 (left) and W43 (right). SINFONI allows to 
resolve the cluster and extract the spectra of many stars. 



8.4. Sky subtraction 

In its most recent release the pipeline corrects for improper sky subtraction; in older re- 
leases this caused residuals in on-off sky-subtracted frames. The sky emission is removed 
from the object observation by subtracting the sky frame from the object frame. This as- 
sumes that the two frames have a stable spectral format. However, it has been found that 
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in on-off, object-sky sequences the instrument setting can occasionally have instabilities 
up to a significant fraction of a pixel. The residual sky features (resembling a P-Cygni 
profile) can be significantly reduced by using an improved data reduction procedure 
developed in collaboration with Ric Davies from MPE (Fig. 8 and [Davies R., (2007)]). 

9. Science Results 

SINFONI started science operations on April 1, 2005, and has since then obtained 
science data for over 1000 hours of science integration time - not including acquisi- 
tion overheads, calibrations or standard star observations. The scientific observations 
discussed below were taken within the science verification runs before the start of opera- 
tions and reduced with the SINFONI pipeline. The results are given as examples for the 
performance of the instrument and the pipeline data reduction. 

[Messineo et al. (2006)] have observed stars embedded in the stellar clusters of the giant 
HII regions W42 and W43 (Fig. 9). In W42 they have resolved the cluster and extracted 
the spectra of 7 stars with high S/N of >30 in H and K bands. This has doubled the 
number of spectral detections with respect to what was available in the literature. In 
W43 the cluster centre has been resolved, and they have detected the photospheric lines 
of several of the 13 brightest stars (9 stars in K and 4 in H band). 




Fig. 10. The discovery of a collimated H2 
jet in M17 (center) is an indication of on- 
going accretion processes. On the left side 
is indicated a possible geometry of the jet. 
On the right it is shown the Br^ map. 



Fig. 11. Star formation in a high-red shift galaxy 
(A370-A5). Observations suggest that it is not a rotating 
disk, and that it is probably characterised by a bipolar 
outflow. 



[Niirnberger et al.(2007)] were studying the formation of massive proto-stars through 
disk accretion of gas in Ml 7. From the SINFONI H2 maps they report the discovery of an 
H2 jet which apparently arises from the suspected proto-stellar source(s) located at the 
very center of the disk. They can infer the diameter and sub-structure of the innermost 
part of the flared disk from the Br^, Br^ and Hel maps. As the ejection of material 
through a jet and/or the outflow is always linked to accretion of gas and dust onto the 
circumstellar disk or onto the central proto-stellar source (s), the presence of a collimated 
H2 jet provides direct and unquestionable evidence for ongoing accretion processes in the 
case of this disk. 

[Lemoine-Busserolle et al.(2007)] (see also [Gillessen et al. (2006)]) used SINFONI to 
probe the star formation of the high-red shift galaxy A370-A5. The clumpy structure 
evidenced by the Ha contour plots are an indication of a stellar component and show two 
extended regions without continuum counterpart (Fig. 11, left). Hq, velocity maps indicate 
the presence of a velocity gradient perpendicular to the stellar emission (Fig. 11, center). 
The Hq, velocity dispersion map shows no indication of the central peak which would 
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characterise a rotating disk (Fig. 11, right). From these observations Lemoine-Busserohe 
et al. conclude that A370-A5 is not a rotating disk, and it is probably characterised by 
a bipolar outflow. 

10. Summary 

The SINFONI pipeline is an important part of the ESO-VLT Data Flow System to 
support SINFONI operations, monitor the instrument's health, generate master calibra- 
tion and scientific products, and assess their quality. Applying the recently improved 
sky subtraction, the master dark, master flat and optical distortion corrections, and the 
wavelength calibration, a good quality 3D data cube is constructed, which contains the 
full spatial and spectral information of the SINFONI FOV for further science analysis. 
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